fix(degreeworks): Provide Polymorphic requirements for majors with specializations by HwijungK · Pull Request #310 · icssc/anteater-api

HwijungK · 2026-02-17T06:55:56Z

Description

We currently save one requirement for a major. However, when majors are taken with a specialization the original major block may change. For example, the Applied and Computational Math major has 2 optional specializations. The major requirement has a "Requirements for Pure Applied and Computation Math" subset, which is removed when specializations are taken.

We now run a major scrape for each major, spec pair. The major_spec_pair_to_requirement table refers to a major code and an optional potentially null spec code, and has a requirement_id that points to another table, major_requirement. To enforce uniqueness on major_requirement, a requirements_hash column is used as the json for requirements is to large to put a unique constrain on.

The Major Requirement Endpoint has been changed to take an optional specializationId (ex = 'BS-201A) along with the programId (BS-201). It is an error to give specializationId as null or to give a specializationID that is not associated with the given majorId.

Related Issue

#263

Motivation and Context

How Has This Been Tested?

Locally tested on graphql and rest endpoints.

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My code involves a change to the database schema.
My code requires a change to the documentation.

…edundancy

…p works properly

…id of specialization table

…that points to seperate major_requirement table

laggycomputer

You also accidentally checked two testing files in, no big deal

laggycomputer · 2026-03-05T22:44:38Z

apps/data-pipeline/degreeworks-scraper/src/components/Scraper.ts

+  majorCode: string;
+  specCode?: string;
+};
+// 'majorName;specCode'. If no specialization is taken, 'majorName;undefined'


Maybe just drop the semicolon and everything after it if there's no specialization. Explicitly leaving "undefined" in a string may be mistaken for a bug.

laggycomputer · 2026-03-06T03:24:36Z

apps/api/src/schema/programs.ts

  }),
+  specializationId: programIdBase.optional().openapi({
+    description:
+      "fetch major requirements when taking the specialization with this ID, if provided; providing no specialization when one is required returns the 'base' major requirements",


providing no specialization when one is required returns the 'base' major requirements"

Too nice of a claim; we can't really say anything about the meaning of the major requirements when a specialization is required, but none is provided...

laggycomputer

closer

laggycomputer · 2026-03-09T19:17:07Z

apps/api/src/rest/routes/programs.ts

        {
          ok: false,
-          message: "Couldn't find this major; check your ID?",
+          message: "Couldn't find this major associated with this specialization; check your ID?",


Grammar is a bit funky here...

laggycomputer · 2026-03-09T19:17:50Z

apps/api/src/schema/programs.ts

  }),
+  specializationId: programIdBase.optional().openapi({
+    description:
+      "fetch major requirements when taking the specialization with this ID, if provided; providing no specialization when one is required results in unspecified behavior and is deprecated",


Suggested change

"fetch major requirements when taking the specialization with this ID, if provided; providing no specialization when one is required results in unspecified behavior and is deprecated",

"If provided, fetch major requirements given this specialization; providing no specialization when one is required has unspecified behavior",

apps/data-pipeline/degreeworks-scraper/src/components/DegreeworksClient.ts

laggycomputer · 2026-03-09T19:19:59Z

apps/data-pipeline/degreeworks-scraper/src/components/Scraper.ts

+          `${schoolCode}-MAJOR-${majorCode}-${degreeCode}`,
+          majorAudit,
+        ),
+        specCode: specCode,


Use shorthand

laggycomputer · 2026-03-09T19:20:55Z

apps/data-pipeline/degreeworks-scraper/src/components/Scraper.ts

+      const inMap = this.parsedPrograms.get(
+        this.asMajorSpecId("Major in Chemistry"),
+      ) as MajorProgram;


This was my goof as we discussed, but you can't as MajorProgram because this is an inappropriate assumption of infallibility in the map get.

apps/api/src/graphql/resolvers/programs.ts

apps/api/src/rest/routes/programs.ts

packages/db/src/schema.ts

June Local.session.sql

Co-authored-by: Dante Dam <laggycomputer@yahoo.com>

…to polymorphic-majors

laggycomputer · 2026-03-23T03:37:00Z

apps/data-pipeline/degreeworks-scraper/src/components/DegreeworksClient.ts

    degree: string,
    school: string,
    majorCode: string,
+    specCode?: string,


More than one optional argument gets a little messy, especially if we add/reorder them. Do you want to pass in some kind of object instead? We could also leave this in if you don't care enough

laggycomputer · 2026-03-23T03:37:59Z

apps/data-pipeline/degreeworks-scraper/src/components/Scraper.ts

      // "optional american chemical society certification"
-      const inMap = this.parsedPrograms.get("Major in Chemistry") as MajorProgram;
-      return inMap ? [inMap[1]] : [];
+      const inMap = this.parsedPrograms.get(this.asMajorSpecId("Major in Chemistry"));


I'd prefer the specialization argument to asMajorSpecId be nullable instead of optional, but that's an opinion.

laggycomputer · 2026-03-23T03:39:35Z

packages/db/src/schema.ts

+  majorId: varchar("major_id")
+    .notNull()
+    .references(() => major.id),
+  specId: varchar("spec_id").references(() => specialization.id),


Should maybe be specialization_id for consistency

laggycomputer · 2026-03-23T03:40:06Z

packages/db/src/schema.ts

+      .primaryKey()
+      .generatedAlwaysAs(sql`jsonb_hash_extended(requirements, 0)`),
+  },
+  (table) => [],


Remove this if we're not going to add any indices, etc

HwijungK added 30 commits January 28, 2026 11:11

refactored RuleBase type to have a nullable proxyAdvice

5336574

added degreeworksSpecRequirment to schema

70da258

added ifElsePart property to RuleBase

1a05141

push DWSpecRequirement when encountering match in ruleArrayToReq

bddc87a

added requireSpec to schema

a761de2

migrated db and did some pg testing

480abb6

updated pnpm

6359bdb

added requireSpec field to graphql schema

cf4e1a0

made requireSpec a nonnullable type

9b396a7

remigrated db after making requireSpec nonNullable

5bcd14e

added comments for evaluating spec req

e8f271c

filtered out DegreeworksSpecRequirement when adding to db to reduce r…

02106bb

…edundancy

removed 2 migrations so that we could redo

b1744ff

reran migration with --name flag

50e0f4f

refactored parsedPrograms to store majorDegrees of (major, spec) pairs

ba20f4f

save found major, spec pairs when finding specs

c7c3997

temp commit, upserting is currently broken

c561897

Merge branch 'main' into polymorphic-majors

ae555a4

encoded parsed Program key into a string instead of a list so that ma…

a32e91a

…p works properly

saved spec_id in major_requirement in coorect - format to match with …

a42d151

…id of specialization table

feat: changed schema to contain major_spec_pair_to_requirement table …

c44cba0

…that points to seperate major_requirement table

Auogenerate id in major_spec_pair_to_requirement table

11c23a5

db migration

f9e8b26

deleted outdated numbered migration files

0fbcf18

Merge branch 'main' into optional-spec

e621ebf

re-migrated

97f7985

made specRequirement a subtype of markerRequirement

bd47eaa

hard coded setting ChemE to have opitonal Specs

2b81a70

refactored major_requirement table to use hashed json as its primary key

1a56be0

removed debug logs used during testing

b67922f

HwijungK temporarily deployed to staging-310 March 5, 2026 21:42 — with GitHub Actions Inactive

used asmajorSpecId to create parsedProgram keys where originally missed

19cb482

HwijungK temporarily deployed to staging-310 March 5, 2026 22:37 — with GitHub Actions Inactive

HwijungK requested a review from laggycomputer March 5, 2026 22:38

laggycomputer requested changes Mar 5, 2026

View reviewed changes

laggycomputer requested changes Mar 6, 2026

View reviewed changes

HwijungK added 2 commits March 9, 2026 10:37

merged main into polymorphic majors

3e78581

doc changes

ac7051f

HwijungK temporarily deployed to staging-310 March 9, 2026 19:12 — with GitHub Actions Inactive

HwijungK requested a review from laggycomputer March 9, 2026 19:15

laggycomputer requested changes Mar 9, 2026

View reviewed changes

minor doc and cleaning changes

730e009

HwijungK temporarily deployed to staging-310 March 9, 2026 20:22 — with GitHub Actions Inactive

HwijungK requested a review from laggycomputer March 9, 2026 20:22

laggycomputer requested changes Mar 9, 2026

View reviewed changes

Update apps/api/src/rest/routes/programs.ts

19cc8d4

Co-authored-by: Dante Dam <laggycomputer@yahoo.com>

HwijungK temporarily deployed to staging-310 March 9, 2026 21:55 — with GitHub Actions Inactive

HwijungK added 2 commits March 9, 2026 14:59

removed local sql file

847e2e3

Merge branch 'polymorphic-majors' of github.com:icssc/anteater-api in…

8c6b5e7

…to polymorphic-majors

HwijungK temporarily deployed to staging-310 March 9, 2026 22:01 — with GitHub Actions Inactive

yet additional doc changes. bless me with english TT

54e5d89

HwijungK temporarily deployed to staging-310 March 9, 2026 22:25 — with GitHub Actions Inactive

HwijungK added 3 commits March 16, 2026 11:39

better error logs on graphql logs and delete migration to sync with main

a4bd0ad

Merge branch 'main' into polymorphic-majors

7947166

remigrate and fix dwClient majorAudit bug

e1c9ddb

HwijungK temporarily deployed to staging-310 March 22, 2026 19:13 — with GitHub Actions Inactive

final i hope

bd21520

HwijungK deployed to staging-310 March 22, 2026 19:38 — with GitHub Actions View deployment

HwijungK requested a review from laggycomputer March 22, 2026 20:01

laggycomputer requested changes Mar 23, 2026

View reviewed changes

	"fetch major requirements when taking the specialization with this ID, if provided; providing no specialization when one is required results in unspecified behavior and is deprecated",
	"If provided, fetch major requirements given this specialization; providing no specialization when one is required has unspecified behavior",

Conversation

HwijungK commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Uh oh!

laggycomputer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laggycomputer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

HwijungK commented Feb 17, 2026 •

edited

Loading